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ABSTRACT 

This paper examines the research on "pull out," a 
method or type of school organization fcx resedial teaching of Title 
I eligible pupils. Four major issues addressed are: (1) the 
educational benefits of pulling students out of the daily routine to 
provide them with' compensatory educaticn services; (2) the impact of 
such action: on students; (3) whether a child is tetter served if he 
Xfjmains in the classroom all day; and (U) altertatives tc "pull out" 
a% ailable for providing compensatory assistarce to ediscationall y 
disadvantaged children. Other related issues examined are: the 
prevalence of "pull out" programs, the benefits and/or .losses 
resulting f rem "pull out" programs^ teacher contact with and 
attitudes toward pulled out pupils, financial costs cf "pull out" 
programs, and the potential contribution cf "pull cut" programs to 
cultural separatism, racial segregation, or even racism. It is 
concluded that despite the near universality oz Exiling Title I 
eligible pupils out of regular dassrccis fcx compensatory 
instruction, the procedure has neither academic nor social benefits, 
may be detrimental, and is; used mostly to satisfy Title 1 
regulations. Alternatives to "pull cut" are recommended. 
(Authpr/BB) 
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SUMMARY 

"PULL OUT" IN COMPENSATORY EDUCATION 

"Pull Out" is a method or type of school organization for remedial teaching 
of TitTeT eligible pupils. With this plan, Title I eligible pupils are pulled 
out of regular classes containing both eligible and non-eligible pupils and sent 
to a different room to receive instruction from a remedial specialist teacher. 
"Pull Out'" has emerged ..as a prominent feature of compensatory education in the 
past few years, and it now concerns policy-makers, researchers, and educators 
alike. This paper was written in response to a request from the Office of the 
Commissioner of E^lication ^o examine the research on "pull out." In the course 
of preparing this opinion, we Interviewed about thirty persons in schools, state 
education agencies, the federal governmeYit, universities, and teacher organiza- 
tions; in addition, we read and, in some instances, reanalyzed data from approx- 
imately 150 documents. . « 

The Incidence and Context of "Pull Out" . , . - 

Roughly 75% of compensatory education pupils receive remedial reading 
instruction in the "pull out" setting; the comparable figures for mathematics 
and language are 45% and 41%, respectively. When these figures, are corrected to 
eliminate pupils in 100% Title I eligible classrooms who do not need. to be 
"pulled out," the "pull out" rates in all othor classrooms rise to 84% for 
reading, 54% for mathematics, and 50% for language arts. When one considers 
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further that pupils might be "pulled out" for one of these subjects and not the 
other, it is. plausible to say that in classes not 100% "Titlk I eligible" the 
practice of " pull out " for compensatory teaching is^ nearly universal . 

"Pull out" is probably more prevalent in small and medium-sized districts 
than 1n large, urban schools. "Pulled but" an'i "mainstreamed" pupils (i.e., 
compensatory education pupils taught in regular classes) do not differ in their 
academic performance before remedji^al teaching; thus, "oulT'out" seems not to be 
prescribed differentially for pupils with varying remedial needs. 

The amount of the entire instructional day spent in the "pull out" setting- 
rose from around 5% in 1973-1974 to around 9% in 1974-1975. The percentage of 
time does not vary by sibject taught (reading vs. mathematics) or by grade level 
(elementary vs. secondary). Although time in the " pull out " setting is a small 
part of the tote* instructional day, at Grade 1 it constitutes almost half of 
the instructional time funded by Title 

^ At the elementary school level, one-fifth of the "pulled out" pupils miss 
regular classroom instruction in the subject for which they are removed from the 
regular class (i.e., they are "pulled out" of regular reading to receive remedial 
reading). One- fourth miss social studies; orie-seyenth miss science. One-third 
miss no academic subject at all since they are pulled out during study periods 
in the regular class, (By some chop logic we do not understand, supplanting is 
not supplanting at all if one supplants science and social studies.) 

The "pulled out^' pupil has three chances in four of receiving remedial 
instruction from a remedial subject matter specialist. (The comparable chances 
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for a mainiitreanied compensatory education pupil are only one in three.) However, 
the "specialist" teachers receive very little training for their job (less than 
ten hours in any one year on the average) and they receive virtually no extrae pay - 
(less than 5% more than regular teachers). These data seem to indicate' that 
remedial specialists are distinguished from regular teachers neither by more 
intensive training nor liy the pay they receive. The most cj(ni-ca.l assessment of 
A ' their role and contribution would be that remedial specialist teachers are merely 
rechristened regular classroom teachers the motive for so ^designating them 
being, perhaps, the need to comply with certain Title I regulations. 

Finally, "pull out" programs appear to be roughly twice as expensive per 
pupil as mainstream compensatory programs, probably because of much smaller class 
size in the former than the latter. 

The Effects of 'Tull Out" 

Experimental evidence is skimpy on the effects of the "pull out" technique 
per se on pupils' academic progress. A study recently published by the National 
Institute of Education (September 1977) alleges to show beneficial effects of 
"pull out" at certain grade levels and in certain subjects and detrimental 
effects elsewhere. We have examined the data and find little support for these, 
conclusions. The academic gains made by "pulled out" and "mainstreanied'' compen- 
satory education pupils in the NIEdata are virtually identical, differing over 
all grades and subjects by less than one-quarter month in grade-equivalent units. 
.Perhaps a better data base for assessing the effects of "pull out" exists in the 
data files of the evaluation of the Emergency School Aid Act (ESAA) conducted by 
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Systems. Development Corporation. There one finds a consistent negative relation- 
ship between the percentage of time pupils spend in the "pull out" setting and 
their math and reading achievement. This relationship was consistent across all 
grades and subjects; it held true for samples numbering about 10,000 pupils in 
total. Moreover, the relationship persisted even after more than a dozen back- 
. ground variables were controlled statistically. 

A vast body of empirical research on instructional methods and organization 
is pertinent to the "pull out" problem because the phenomena investigated share 
various features with the "pull out" technique. Such related topics include the 
following: (a) ability grouping, (b) mainstreaming the handicapped, (c) racial 
desegregation, (d) labeling pupils' with consiequent changes in teachers' expecta- 
tions of them, and (e) peer tutoring. Our synopses of the research evidence on 
these topics are as follows, (a) The research on ability grouping is inconsis- 
tent, uninforniative, and a battleground for various social ideologies; it was 
not helpful to us in forming opinions about "pull out." (b) The research on 
mainstreaming the handicapped was exceedingly skimpy, but the findings of three 
studies point toward the benefits of i^ntegrating EMH, EMR, or emotionally dis- 
turbed pupils into the regular classroom, {c) Nearly all research on racial 
desegregation fails to trace racial mixing at levels lower, than the school 
building. Using the Coleman data, McPartland (1969) assessed the effects on 
verbal achievement of black pupils having predomin?iitly black classmates instead 
of white classmates. Although the offect diminished as more background variables 
were partialed out, the effect of racial segregation at the classroom level was » 
negative in first analyses and never appeared beneficial regardless of how many 
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variables werie statistically contiolled. (d) Research into the effects of 
labeling pupils on teachers' behavior toward them and expectations of them 
proved to be most pertinent and startling. Labeling a piipil "mentally retarded, 
"intellectually slow," or "academically weak," reduces his academic achievement 
(by one-quarter standard deviation below that for comparable pupils not so 
labeled). Furthermore, teachers' attention and support for pupils invidiously; 
labeled are reduced by one-third standard deviation (below those iFor comparable 
unlabeled pupils); and teachers' judgment of labeled pupils' success, motivation 
social, competence, etc. is reduced by nearly one-half standard deviation. These 
findings from over forty experiments indicate that the effects of labeling . 
pupils are large and worrisome. (,e) Finally, research on peer tirtoring, which 
presumably could occur less often in the "pull out" programs. Pupils pulled out 
of regular classrooms would have to receive remarkably effective compensatory 
programs to offset the potential risks incurred. In our opinion, the ''pulled 
out" pupil is placed in moderate jeopardy of being dysfunctional ly labeled-, of 
missing opportunities for peer .tutoring and role modeling, and of being segre- 
gated from pupils of different ethnic groups. 

I 

Historical and Political Context of "Pull Out" 

We believe that the "pull out" problem was created by the ESEA Title I 
regulations and the manner in which they have been interpreted and enforced. To 
•juote one state education department official ," 'Pul 1 out' exists for one reason 
only; because the 'locals' are afraid Big Brother will catch them in a 
'supplanting' violation." The practice of pulling Title I eligible pupils out 
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of regular classrooms so that a "speciaiist" teacher could give them instruction 
in a'ieparate classroom did not grow put of professional judgment about curric- 
ulum or instruction. The history is complex, but nearly any disinterested 
reading of it leads to the same conclusion: "pull out" is an artifice created 
by schools at the urging of USOE's minions in state .education departments to 
satisfy regulations concerning "supplementing, not supplanting" and "excess 
costs." The regulations themselves reflect a philosiophy that seems seldom to 
have been seriously challenged. They are enforced with a nearly obsessive con- 
cern that"a."noneligible" pupil might receive Title I services. Yet the argument 
can be made that even pupils performing 3t grade level and above are education- 
ally deprived by merit of attending a school with large concentrations of poor 
(since such schools attract less qualified teachers, have poorer opportunities 
for peer tutoring, etc. ). ^' ^ 

One finds virtually no support for the "pull out" concept among educators or 
their professional organizations. Teachers worry that pulling pupils out of 
class creates discontinuities in their schooling and makes coordination of 
teaching difficult. Others worry that the regular classroom teachers' wil 1 feel 
less responsible for pupils whose needs are presumably being met somewhere else 
by a specialist teacher. The National Education Association regards "pulT out" 
as a mtmor issue and will merely watch its evolution., being concerned only with 
keeping pupil -to- teacher ratios low. Th<^ "pull out" problem seems to be no one's 
rpajor concern. But it may well be ^e of those quiet, inconspicuous matters' that 
count heavily in ways seldom clearly seen. 
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Conclusions, Observations, and Recommendations 

Our work has led us to the followirjg conclusions and observations about the 
"pull out" technique and several recomir,endations for dealing with the problems 
it raises. 

1., .Pulling Title I eligible pupils out of regular classrooms for compensa- 
tory instruction is virtually universal. 
'.• ' ' ^ 

2. The "pu1T out" procedure per se has no clear academic or social benefits 
anri may, in fact, be detrimental to pupils' progress and adjustment to 
scl.ool . 

3. The "pull out" procedure is used by schools more to satisfy Title I 
regulations than because it is judged by teachers to be a sensible and 
beneficial plan. 

We wish to bring the following recommendations to the attention of those 
persons at all levels who administer Title I programs and who will influence the 
. evolution of cCinpensatory education: 

1. The Title I regulations, which now reflect an overweening concern with 
targeting funds on "eligible" pupils, sjTOuld be examined. New consid- 
erations should be given to the needs of all pupils in poor schools and 
the integrity of total school programs. 

2. Instructional strategies should be devised that would eliminate the 
invidious labeling of compensatory education pupils and their segrega- 
tion from classes of "regular" pupils. 

3. Teachers, administrators and other persons connected with Title I pro- 
grams should be 1nfom>id of the findings of research on the "pull out" 
method and associated phenomena. 

4. Methods should be devised of counteracting the possibly detrimental 
effects of "pull out" where educators choose to use it or have no 
reasonable alternatives. Such methods could include means for coor- 
dinating instruction across two sites and techniques of teacher obser- 
vation that lessen the possibility that "pulled out" pupils will be 
unconsciously neglected in regular classes. 



Chapter 1 
THE ISSUES 



This paper was commissioned for the purpose of answering four questions: 

1. What are the educational benefits of pulling students out of the daily 
routine to provide them with compensatory education services? 

2. How much of an impact on the child is there? Positive or negative? 

3. Can we better serve the child if he remains in the classroom all day? 

4. What are alternatives to "pull out" available for providing compensatory 
assistance to educationally disadvantaged children? 

These questions are encompassed by a slightly broader set of issues around 
ch we organized our inquiry and this report: 

1. How prevalent are "pull out" compensatory education programs? 

a. What kinds of schools use them? 

b. From what types of regular classroom activities are pupils "pulled 
cut"? 

c. What services are given to pupils in "pull out" programs? 

2. What benefits or/losses result from "pull out" programs? 

a. Are pupils placed in richer learning environments than can be 
found in regular classrooms? 

b. Do pupils Jose the benefits of being with clasr-mates not similarly 
"pulled out"? 

c. Are pupils labeled as a result cF being "pulled out"? And does the 
label cause them to be treated differently by teachers and class- 
mates? 

d. Do teachers lose cantact with "pulled out" pupils and feel less 
responsibility for their progress? 

e. Is the academic progress of "pulled out" pupils accelerated or 
retarded? 

f- Does "pull, out" contribute in unintentional and subtle ways to 
^. cultural separatism, racial segregation or even racism itself? 

3. What are the financial costs of "pull out" programs and how do they 
compare^ with other ways of providing compensatory instruction? 
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46 What position on "pull out" should KEW/OE take?. 

a. Who is responsible for "pull out" programs? 

b. What alternative policies are there? What benefits and costs are 
associated with each? What would be the likely consequences of 
recommending each? What political/social forces support cr oppose 
each alternative? 
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Chapter 2 
METHODOLOGY OF THE STUDY 



The issues in this study were researched via two principal, avenues: (1) 
interviews with educators, policy-makers, and researchers, and (2) examination 
and synthesis of documents on "pull out" programs, the policy context from. which 
they arose, their character, and their effects. 

Interviews ^ 

Approximately thirty individuals were interviewed, most by telephone, in 
the course of this study. Opinions and observations were sought from persons at 



'^laTT^'Te^^ to state departments of education. 



to local school district?. /Researchers, practitioners, and policy-makers were 
interviewed. Tiie.names of those interviewed who would- consent to have thei r 
names cited appear in Appendix A. 

Documents 

Over 150 documents were examined dn the "pull out" issues. They fell °into 
three general categories: (1) documents relating to' the legislative history. and 
administration of Title I of ESEA, (2) reports of empirical research on the char- 
acter anjcUe^ects of Title I "pull out" programs, and (3). reports of research on 
educational phen^t^na related to the "pull out" instructional technique: 

1. History and Administration of Title I (ESEA). The standard references 
(e.g.. Bailey and Mosher, 1968; McLaughlin, 1975) were useful in analyzing the 
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political forces that are responsible for the creation of "pull out*' programs. 
These issues will be examined in Chapter 5, 

2. Empirical Research on " Pull Out ," Empirical research specifically o n" 

the problem of "pull out" is limited. Only three studies can make any pretense 
to be representative of more than extremely limited regions: the NIE survey of 
compensatory education and the Systems Development Corporation studies of the 
Emergency School Assistance Act and of the "Sustaining Effects of Title I." All 
three studies contain survey data ou the incidence of "pull out,".. and each will 

eventually attempt to advance quasi-experimental evidence on the effects of the__ 

"pull out" technique. 

Comparability of findings of the NIE and the Systems Development Corpora- 
tion's ESAA surveys is an important feature of Chapter 3 where what is known 

" • about- the-character'Brid impart^ is presented. The ESAA 

evaluation focused on elementary and secondary scjiools undergoing racial desegre- 
gation or with large concentrations of minorities which could not be reduced. 

. Elementary schools were designated either Basic or Pilot; Pilot schools were 

■j> ■ .. . . 

minority concentrated (65% black and IS^^^-SpahishJ whereas Basic elementary^ 
schools were more nearly nationally representative. ESAA Basic elementary 
schools enrolled 40% black and 85^ Spanish surnamed' pupils; the -compensatory ed!J- 
cation pupils in the NIE: survey were 35% black and 10% Spanish surnamed (although . 
they were, enrolled in districts that were 20% black and 6% Spanish surnamed. 
Where possible, NIE findings will be compared only with the ESAA findings for 
3asiq. elementary schools. The ESAA evaluators reported that their sample 
slightly overrepresented "large and medium-sized" school districts compared to 
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the nation as a whole; any sample o ^tle I districts would do likewise. Exact 
comparability of samples is unnecessary where one is comparing relationships 
among variables (as opposed to averages and other measures o|?level). Since the 
major comparison of ESAA and NIE findings will concern regression coefficients, 
the comparability'^f the samples appears adequate for our purposes. 

A very limited number of experimental studies of the effects of "pull out" 
are available. Although they are sparse and scattered, they are important pre- 
cisely because there is so little controlled study of the effects of "pull out" v 
programs that one can use to support or question the quasi -experimental evidence 
from the large NIE and SDC^ surveys. , ^ — - — ' ^"^ 

3.- Research on Topics Related to " Pull Out ." The "purl out" technique 
shares features with several related instructional methods and organizational 
schemes: ability grouping, "mainstreaming" the handicapped, labeling and teacher 
expectations, racial desegregation, etc. The research literature on each of 
these related topics were searched or else we relied on reputable reviews to form 
an opinion. For example, the empirical research literature on teacher expecta- 
tion bias (the effev.cs of labeling pupils) was pertinent and had not been 
reviewed adequately for our purposes; so we obtained over forty research reports, 
read them, and reanalyzed most of the data. On the other extreme, the literature 
on ability grouping is voluminous and ancient, and it has been frequently 
reviewed; we relied on the findings of these reviews^ 

References to all documents consulted appear in the bibl iogra.phy at the end 
of this report. 
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THE NATURE AND EFFECTS OF "PULL OUT" PROGRAMS 

Survey data gathered specifically to examine the "pull out" problem are 
skimpy and widely scattered. The data that follbw were pieced together from a 
variety of sources reporting surveys conducted at different times on different 
populations. Nevertheless a coherent picture emerges . 

Incidence of "Pull Out" and Its Context ' 
. - „-_The-N4£~survey-reveatetf "thlit~ab"ouF^7^^^^^^ educati on pupi Is are 

"pulled out" of their regular classroom to receive compensatory reading instruc- 
tion. The comparable data for cornensatory language arts and mathematics 
instruction are rates of "pull out" of 41%and 45%, respectively. The NIE data 
are quite in accord with early results from the "Sustaining Effects" study being 
conducted by Systems Development Coir*poration that show 77% of the compensatory 
education pupils bsing "pulled out" of regular classrooms. for reading or mathe- 
matics instruction. (In the "Sustaining Effects" study, 24% of the pupils 
received special instruction in the regular classroom from a specialist teacher.) 
It must be noted, however, that there is an appreciable incidence of classrooms 
composed entirely of Title I eligible pupils for which "pull out" is unnecessary. 
Hence, tivi incidence of "pull out" programs where they are possible is even 
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larger than the above figures might lead on e^to believe. By combining data from 
different tables in the NIE survey, we have determined that the rates of " pul 1 
9Mt" for compensatory education pupils not in^ 100% eligible classrooms are as^ 
follows : 

Subject ^ ' Pull Out Rates 

Reading .84% 
Language Arts ' 50% . 

Mathematics 54% 

(The comparable rate for either reading or math "pulj, out" estimated from the 
"''Sustaining Effects" survey is 90%.) The " pull out" organization is virtually 
universal wher p com pensatory and regular pupils are mixed ijx class.'ooms . 

Use of the 'pull out" method is probably more prevalent in small and rural 
schools than in large, urban schools because of the heeivy concentration of com- 
^; ,, -.p^^ pupils in the latter. Because of the definition of "Title I eligi- 

bility" adopted by smny districts, it is not uncommon'to find many urban schools 
composed entirely of Title I eligible pupils. Such is the case in large cities 
like Dallas-, Chicago, Philadelphia, New York, etc. \ 

, The amount of instructional time that compensatory education pupils sp^-^d in 
"pull out" programs can be pieced* together from various sources in the SDl 
reports pf the ESAA evaluation. T-able 3.1 contains the percentages of all. 
Instructional time that compensatory pupils spend in the pull out setting and 
relates that rate of time to the subject taught, the grade level of -the pupils, 
and the datftof the survey. 

Several observations can be made on the findings in Table 3.1: (1) instruc- 
tional time spent in the "pull out" setting is a small part of total time 
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Table 3.1 

Percent of Instructional Time in "Pull Out" Settiig 



Reading 
Math 



Elementary School 

Pilot Basic 
"(Minority 
concentrated) 



Secondary School 



1973-74 1974-75 


11973-74 1974-75 


1973-74 1974-75 


5.7% , 


8.7% 


6.4% 


8.1% 


4.1% 


8.0% 


4.8% 


8.8% 


3.2% 


8.2% 


5.2% : 


9.2% ; 
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compared to "independent seat work*' at 10-20%, and "group instruction" also at 
10-20%, both in the regular classroom; (2) the "pull out" setting is being used 
increasingly 75% more instructional time was spent in the "pull out" setting 
in 1974-75 than in 1973-74 (8,5% vs. 4.9%); (3) there is no relationship between 

s .... • ... 

the amount of time in the "pull out" setting and the minority concentration 
(Pilot vs. Basic) of the school at the el ei?entary level , nor between the Amount 
of "pull out" time and the level of the school (elementary vs. secondary); (4) 
time spent in the "pull out" setting is roughly the same for remedial reading 
and math. However, these data that tend to assuage concern about "pull out" and 
make it seem an insignificant phenomenon, should be set over against findings . 
from the District Service No. 1 data set analyzed by the National Opinion 
Research Center and communicated to us by David E. Wiley (26 September 1977, 
personal communication) : at GradeH l ," l)f "a1 T readin g j nstruction funded by Title 
I^, 38% of the time is in the " pull out " setting ; of al 1 non-readfng instruction 
funded . by Title. 1 , 48%. of the time is in the " pul 1 out " setting . Figures from 
an earlier study by Kiesling (1971)-agree with these findings, Across 37 Title 
. I programs in California^ it was found that an average of 45% of instructional 
time funded by Title I occurred outside the regular classroom (standar'd^ devia- 
tion = 35%). ^ . 

Little is known about what types of pupil are designa^ted for "pull out" ^ 
instruction isnte.ad of mainstream comp6tisatory instruction in the regular class- 
room. One scrap of evidence is^ avai labile from the NIE (September 1977) report 
^ ' . ■ .... 

on "Effects of Services": the pretest ( Fal 1 , grades 1 and 3) achievement of 
mainstreamed and pul led out pupil s is virtually identical . It does not appear 
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that the "pull out^nethod is being selectively prescribed at least it is not 
being prescribed on the basis of pupils' initial achievement level . 

The character of the "pull out" programs concerns what compensatory pupils 
leave behind in the regular classroom when they are "pulled out" and what they 
find in the "pull out" setting. 

When an elementary school compensatory education pupilj's "pulled out" of 
bis xegular class room. for remedi ^ ins truct i on i n read ing^^ language or mat h , 
the chances are: 



1 in 3 that no academic instruction is missed. 




1 in 5 that the same subject for which he is pulled out is missed, 

1_ in[ 4 that instruction in social studies is missed , 

1 IP Z that instruction in science is missed . 
The " pulled out " pupil has better than 3 chances in 4 of receiving remedial 
instruction from a subject matter specialist . A " mainstreamed compensatory / 
education pupil has only about chance in^ 2 receiving remedial instruction 

from a subject matter specialist. / 

. ^ 

What kind of training do such ^special 1st teachers have?' There rnay^be some, 
clue la the ESAA (Year #3) report (Coulson et al., 1977). There, the 
"specialist" teachers are called "remedial" teachers. On the average, ESAA 
elementary school teachers: who received any training at all received about 12 
hours in-servide training in teaching reading and about 15 hours in-service 
training in math during the year of the survey. Teachers in regular classes 
reported receiving about half as many hours of ip-service trainin.g in reading 
and math during the same time. However, 13% of the remedial reading and 25% of 



the remedial math teachers reported having received no in-service training during 
that year; 12% of the specialist math teachers reported never having received 
special training. 

It is, perhaps, more revealing to inquire into the value attached to the 
specialist teacher training by the schools '- such value being revealed in the 
increased s/lary of specialist teachers that the schools are willing to. pay. The 
ESAA (Year #3, p. A-Z-J) report reveals that*^ regular classroom teachers in second- 
ary and elementary _5ch^ols were paid $9-45_ hourly, whereas reiTjJdial reading 
.njath specialist teachers were paid $9.95 hourly, a differential of only 5%. 
Hence, however the schools may regard the special expertise and training of 
remedial specialist teachers, such regard is not reflected in a significantly 
higher sajary. 

These data seem to Indicate "that 'remedjal specialists are distinguished, 
neither by their obviously superiur training nor by the pay they recel.ve for 
their services. The most cynical assessment of their role and contribution 
would be that remedial specialist teachers are merely rechristened regular class- 
room teachers -- the motive for so designating them -being the need to comply with 
certain regulations. JTjjfio, studies, however, are alleged to show that the use of 
remedial reading specialists is consistently related to reading gains (Kiesling, 
1971; Flynn, Hass, Al-Salam, 1976). We have not had time to obtain and examijie. 
these two studies; but the key question about them is >7hether they were con-~ 
trolled experiments or quasi -experiments and if the latter, how well they were 
performed. . v 

Finally, the per-pupil cost of remedial instruction in the "pull out" 
setting is roughly twice the c6st of such instruction in the regular classrpom 
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(Dieneiiiann, et al. , 1974). The cost difference has less to do with increased 
pay for specialist teachers, as was just seen, than with the smaller size in 
"pull out" settings. /. • 

The pjpil "pulled out" of the regular classroom is placed in a smaller 
class that is probably ethnically and racially more homogeneous than the regular 
classroom. The average class size for remedial reading and math instruction is 
abbyt 10 pupils although the regular classroom has an average of about 27 pupils. 
J nsjrjjcjaon-i^^^ iP--.b.eIs Ji 3h^^^^ 1 i zed _ . 

as might be expected where class .size is reduced (NIE, September 1977, Appendix 
B), No data are available on the relationship between pirticipati on in "pull 
out" programs and race^or ethnic group membership. It/{ias long been known that 
classrooms in Title I schools are far more racially heterogeneous than the 
schools in which they reside." For example, the 1969 Survey of Compensatory 
Education showed that although only 13% of .black pupils attended elementary ;^ 
schools that were nearly totally black (90-100%), 71% of black pupils were placed 
in, nearly totally black classes (Glass, et ai.„ 1970, p. 20 and p. 36). We do . 'i^ 
not know the comparable figures for 1977, but ^certainly the extensive use Of 
"pull out" programs does nothing to promote the racial integration of classrooms. 

The Effects of "Pull Out" 

Evidence on the effects of the "pull out" method £er se on pupil achievement 
is scant artd derives principally from two sources: (1) the NIE survey of compen- 
. satory education (NIE, September 1977); (2) the SDC evaluation of the Emergency 
School Aid Act.' - " ' ' ' 

o . ■ ■ • ' / '■/■■' " ■ 
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The NIE study is the most prominent. Its findings are summarized in Table^ 
3.2. The conclusions drawn in the NIE study from inspection of Tablo 3.2 are as 
follows: 

"Results differ by grade level. First-grade students 
who received instruction in the mainstream setting made sig- 
nificantly larger gains in both reading and mathematics than* 
those in pullout settings. In the third grade, however, these < 
findings are reversed im mathematics , and the two settings are 
almost equal in reading. It appears that neigher setting is 
consistently associated with greater instructional effective- 

considered, the interpretation of the differences in gains under 



"mainstream" and "pull out" in Table 3.2 says more about them than we would have 
said. The table conveys a single impression: there are no important differ- 
ences in performance between "mainstreamed" and "pulled out" pupils. 

The NIE effects study was not experimental; compensatory pupils were found 
in regular or "pull out" settings as they naturally occurred. It almost goes 
without saying that the two groups would not be rar.dbmly equivalent at the out- 
set; considering this fact, it is surprising that the groups are as equivalent 
as. they appear to be on the Fall pretest. Jhe equivalence apparently reflected 
in the Fall pretest at grade 1 may be illusory; scores as low ^s those obtained , 
could reflect chance performance or the inability of the test to discriminate 
because the pupils were below the effective measurement "cellar" of i"he test. 
In tnd»: case, the apparent pretest equivalence of the two groups might di.sappear 
if the pupils were measured with a test more appropriate tg tjieir level . 

The presumption is strong that the protests in. ^he NIE study were so diffi- 
Cfjlt for all pupils tjiat the dbtained scores are merely chance level and,' hence, 
one can have little Qonfideric'e that the "'pull o.tit" and "mainstream" groups were 

" , : . . / ,■■ ;■ . 

' ■ . . J 

- ' . " f^/^ ' ■ - 
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Table 3.2 

Fall and Spring "Comprehensive Test of Basic Skills" (CTBS) 
Scores for "MainstVeamed" and "Pulled Out" 
Compensatory Education Pupils* 



Grade 1 

Reading ■ 

Mainstreamed 
Pul-led Out 

Mathematics 

Mainstrdamed 
Pulled Out 

Grade 3 

Reading ^'-v 

Mainstreamed 
Pulled, Out 

Mathematics' 

Mainstreamed 
. Pulled Out 



No. of 
Pupi 1 s 



311 
1,044 



Approx. 
No. of 
CI ass rooms 



90 
90 



1,172' 90 
257 "<40 



195 
1,211 



170 
622 



? 
90 



? 
45 



Average, 
Raw Scores 



Fall 



24.3 
23.6 



15.7 
15.6 



19.4 
19.8 



20.5 
21.5 



Gain to 
Spring 



27.0 
22.7 



12.1 
8.8 



10.0 
11.4 



12.0 
16.3 



Average Grade 

Equi val ent 

Scores 



Fall 



0.4 
0.4 



0.4 
0.4 



1.8 
1.9 



1.6 
1.7 



Gain to 
Spring 



1.3 
1.2 



1.2 
1.0 



0.5 
0.6 



0.9 
1.2 



* After Table 2, p. 23' pf NIE (1977),r 



initially equivalent in other important respects. The pretest raw score reading 
mean at grade 1 is about for both group^s. But Level B of tht CTBS has 84 
reading items of which 46 have three-options and 38 have four-options. If pupils 
guessed randomly among the options, a chance raw score would be expected to equal 
about 24.8, nearly identical to the reading pretest averages. If pupils guessed 
randomly on the 56 three-option rrjath items of CTBS Level B, their expected 
ayerage wouTd be 18!^ 7, a figure' abviut thr^e raw-score points about the bfase^^^^ 
raw-score means in Table 3.2. We suspect, then, that CTBS Level B ms Inappro- 
priately difficult as a pretest and that the pretest means are dubious evidence 
of equivalence of the "pull out" and "mainstream" groups.. 

Regardless of the true, but unknown, initial -non-equivalence of* the main- 
streamed and pulled out groups, no attempt wa^made^in the NIE-study to measure 
personal or environmental variables and correct posttest achievement accordingly.. 
It is doubtful that if such "attempts had been made that it would have madfe much 
difference. . 

. It is exceedingly difficult to assess the statistical significance of the 
NIE findingsM'n Table 3.2. Clearly, the individual pupil ought not to be , 
regarded as the unit of statisticc^l analysis. However, it is impossible to tell 
from the report precisely how many intact groups are represented in the' samples 
of several hundred "pull out" and "mainstream" pupils. We. have attempted a few 
"qutck-and-dirty" analyses assuming about ninety classrooms of each type of , 
pupil, and in most instances, the posttest means appear to be statistically 
significantly different. However, we have little confidence in these calcula- 
tions. 



It is dubious to attempt to read statistical significance or psychological 
importance into the trivial differences in gains between ^"mainstream" and "pull 
out' programs in Table 3.2. The overall finding averaged across grades and sub- 
jects is the most reliable, and it reveals virtually no difference between 
achievement in mainstream and pull out settings: the NIE study x shows the differ- 
ence in achievement gains between mainstream' and pull out programs to be one- 
fourth of one month 1n grade-equivalent units . 

The third year ESAA evaluation report (Coulson, et al_. , 1977) presents more 
believable evidence on the effects of "pull cmt" programs. The analyses in 
question are complex and difficult to describe; but briefly, the design was as 
follows: the achievement scores for several thousand pupils in reading or math- 
ematics at grades* 3-5 and 10-12 were regressed onto several independent variables 
including various adult to pupil ratios, total hours of instructidn, pupil 
absenteeism, teachers' expectations for pupils' performance, etc' andv "proportion 
of time spent in either 'pull out' reading or math." Thirty-three separate step- 
wise regression analyses were performed. Tfie regression weight for the "propor- 
tion of time^in 'pull out'" can be regarded as indicative of a quasi -experimental 
effect (at least on a' par or superior in val idity to-the^mean differences in the 
NIE study). In 33 analyses*, the *'pul 1 out" variabl e entered the regression-- 
equation eight times; each time it was negative and statistically significantly 
so. The, size of the rion-zero weights did not vary by subject (reading arid math) 
or grade 1 eve 1 . The^ interpretation is tiiat the greater the proportion of time 
Che pupi 1 spends in " pull out" reading or mathematics , the lower his actiievement . 

. . ; -y \ 

* 

Appendix VII-B through Appendix. VI I-M in Coulson, et al. (1977). 



• - 24 - 

One similar study (Kieslim, 1971) carried out on data from 37 Title I pro- 
jects in California, regressed achievement onto several "input" variables 
descriptive of compensatory program resources. The input-variable "percent of 
time in regular classroom" (the inverse j)f the "pull out" variable in the ESAA 
study) carried a small negative weight (the opposite of the ESAA finding) in the 
regression analysis, but the weight was not statistically significant. 

Satisfactory experimental and quasi -experimental studies on the effects of 
"pull out" are still needed. However, one finds nothing in the studies reviewed 
above that points to benefits of the technique; and one study (the ESAA evalua- 
tion) raises suspicions that the effects of "pull out" may be detrimental to 
academic achievement. 

..: '.. •(. 
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Chapter 4 

EFFECTS OF "PULL OUT" INFERRED FROM RELATED RESEARCH 

As was seen in t|ie previous section, there is little direct and represen- 
tative research evidence on the effects on pupils of "pull out" compensatory \ 
programs. The "pull out" program form shares features with several other methods 
or phenomena of teaching: "mainstreaming" the handicapped, -labeling and conse^ 
quent teacher expectations, ability grouping, peer tutoring, and racial desegre-:, 
gation- Some empirical research exists on each of these related topics. From \ 
it, a plausible argument might be constructed about the possible effects of 
"pull out" programs on pupils. <~ ' 

"Mainstreaming" the Handicapped 

Int grating retarded and emotionally disturbed pupils into regular class- 
rooms is a visible and galvanizing educational concern. It is ofter spoken and 
written about, but seldom studied in any disciprtned way. The phenomenon is 
pertinent to the "pull out" question in, several respectSv .= As would be;true if 
"pull out" programs were abolished, "(^tainstr-eamed" handicapped pupils are moved 
from separate resources rooms into larger groups, of' higher achieving pupils with 
regular teachers. The important difference between ::TiTainstreaming speci 
tion pupils and reversing "pull out" for compensatory education pupils is that 
special education pupils are typically worse off and harder to teach than com- 
pensatory education pupils. 
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The empirical research base for "ma instreaming" retarded or emotionally dis- 
turbed pupils is a meager four documents. What might have been expected to have 
been the most comprehensive and significant study on mainstreaming proved to be 
an expensive disappointment. The OE Bureau of Education for the Handicapped 
study^of mainstreaming entitled Project PRIME is uninformativedon questions of 
the relative advantages of mainstreaming versus special placement. Martin J. , 
Kaufman of BEH was kind enough to let us see a draft of the Project PRIME report. 
The study eschewed a^ny comparison* of mainstreamed and non-mainstreamed jDupils etnd 
justified this choice as follows: "The nature of the sampling procedure^ pre-:.,.; 
eludes making comparative statements about the relative social competency of EMR 
and normal children, or mainstreamed retarded children, and retarded children in,v 
segregated classes. The wide disparity of characterist'its of these three samples 
of children makes any comparative statement virtually uninterpretable. Further- 
more, in view of recent federal legislation (PL 94-142), statements comparing 
regular and special class placements may be less meaningful than statements con- 
cerning which aspects of the classroom environment maximize the social competence 
of retarded pupils." At another point in the report, it is written that; "Given 
that mainstreamed. education is now mandated by law, it is no longer a practical 
issue to determine whether mainstreamed or segregated education provides a more 

advantageous environment to retarded children's sociaT competence " The. 

assumption that PL 94-142 mandates mainstreaming is patently.^ false: it irequires 
only that retarded P'>pils be placed in the "least restrictive environment" and 
the int'erpretati^.1 of the phrase is left to the local district. The BEH report 
on Project feiME presents a plethora of virtually uninterpretable multiple 



correlations (presumably having to do with "optimizing" circumstances for main- 
streamed pupils). It is singularly lacking in quasi-experimental ingenuity and 
completely uninformati ve on the question of the wisdom of mainstreaming. 

Three classic, small-scale studies form the decrepit empirical foundation 
of mainstreaming policy. The oldest study by Vacc (1968) is widely believed to 
lend research support to segregated classes for the emotionally disturbed. It 
doesn't. The study was confounded by differential regression effects for the. 
segregated and mainstreamed pupils and the data analyses were poorly done. Our 
reanalysis shows no evidence of differences in achievement or emotional adjust- 
ment between the mainstreamed and segregated pupils. Two related studies 
(Gampel, et al_. , 1974; and Goodman, et a^. , 1972) lend empirical support to the 
advantages of mainstreaming. The experiments were similar in that educable 
mentally retarded (EMR) pupils were randomly assigned to segregated and main- 
streamed classes. The data show convincingly that the behavior of the main- 
streamed EMR pupils came to resemble that of their normal classmates and 
increasingly had less in common with the behavior of their segregated peers. In 
general, segregated EMR pupils behave in more socially awkward and hostile- 
aggressive ways than do mainstreamed EMR's. 

Thts thin soup of rese^.rch is hardly the kind of diet upon which one can 
build an innovation or a policy; but what there is of it, certainly does not 
refute the wisdom of mainstreaming. 
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Research on Labeling: Teacher Expectations 

The largest and most methodologically sound body of research relevant to the 
"pull out" issue concerns the effects of labeling pupils. This line of inquiry 
is known among educational rfese^a^^chers as the study of "teacher expectations" or 
the "expectancy bias," Most of it dates from Rosenthal's original investigation 
of the "Pygmalion effect." 

The paradigm for research on this topic is as follows: an experimenter 
Informs a teacher that a subgroup of her class shar6s a certain attribute, most 
often high or low intelligence or learning potential ; in fact, the subgroup was 
only randomly deterji'ined so that subsequent, changes in teachers' behavior toward 
the pupils and the pupils' progress can be attributed to the label per se . 
• Research on "teacher expectations" (or "labeling") is pertinent to the "pull out" 
• issue because both labeling of a pupil as being "Title I eligible" and empha- 
sizing such a lable by removing him from the regular classroom could create 
strong expectancy biases in teachers and pupils alike. 

We were initially skeptical of the significance of the "label ing-expectation" 
with IQ. After reading and analyzing data from a few dozen research reports, 
however, we have come to recognize the issue as being of central importance and 
deserving serious attention. Our conclusions, to be detailed shortly, are 
these: to label a pupil as^ "slow," "dull ," " low I^^e" or " lacking potential" 
works almost no effect on tns IQ^, a modest but measureable effect on his academic 
achievement , an^ a_ substantial effect ' on his teachers ' opinions of him and^ how 
they treat Kim . 
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Forty- three studies of the effects of labeling and teacher expectancy were- 
located.* These studies were read to determine the form of the label or expec- 
tation, the adequacy of the research design, the subjects studied, and the 
measure of effect. Three general categories of effect were distinguished: (1) 
effect of label or expectation on teacher judgment attitudes expressed or 
ratings given by the teacher concerning the students' level of achievement, 
potential, or behavior; (2) effect of label or expectation on teacher behavior — 
praise, criticism, time spent, and learning opportunities provided; (3) ieffect of 
the TabeJ or expectation on the students' IQ, classroom achievement, etc. In 
order to integrate the findings of the studies within these categories, the 
statistical results were converted to "effect sizes" expressed as the difference 
between the mean of the more favored group (with high expectancy or label) and 
the mean of the less favored group, divided by the'standard deviation of the less 
favored group, that is ES = (5^ ^igh ' \ow^/^Low* ^" effect size of +1 

indicates that a person at the mean of the low expectancy or labeling condition 
is worse off than 84% of the persons in the high expectancy group. (For an . 
explication of the statistical procedure, see Glass, 1977.) 

Table 4.1 contains the results from this meta-analytic procedure. In the 
category Teacher Judgment, eleven controlled studies produced 33 effect sizes, 
the average of which is .47. Thus, the high expectancy ar lable had almost one- 
half of a standard deviation effect on teachfer justments.^. Teachers' ratings of 
the pupils* intellectual ability showed the' most influence of the labeV: ETs. = 
.93. Teachers rated the intellectual ability of students in the high expectancy 

For the references, see the bibliography in Smith (1977). 
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Table 4.1 
Results of the Meta-Analysis of 
43 Labeling/Teacher Expectancy Studies 



Category 
of Effect 

Teacher Judgment 



Teacher Behavior 



Student Effect: IQ 

Student Effect: 

Ac))ievement 

/ 

■ / 



/ 



other Student Effects 



Sub-Category 



Rated Achievement 
Rated Intellectual Ability 
Rated Social Competence 
Rated Motivation/Interest 
Appeal /Inclination to Teach 
Behavioral Ratings 



Praise/Support/Sustaining 
Feedback/Reinforcement 
Attention/Time Spent 
Learning Opportunities 
Criticism/Antagonism/ 
Negative Interaction 
Ignore/Withdraw From 
Other 



Number of 
Effects 



33 



Reading 
Math 

In-Cl ass/Grades • 
Words, Concepts Learned 

(Lab Tutorial ) 
Other Achievement 



Creativity 
Sociometric Choice 
Social Competence 
Attitudes 



49 



25 



41 



12 
4 
2 
3 
2 

10 



17 
12 
6 

7 

•'5 
2 



10 
9 
5 

2 

15 



1 
3 
1 
2 



Average 
Effect Size 



.47 



,32 



14 



,26 



16 



.26 
.93 
.75 
■.02 
.03 
.72 



.01 
.38 
1.09 

.21 
.52 
.14 



.54 
-.07 
.42 

1.11 
.10 



■.03 
.10 
.89 
-.02 
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group almost one standard deviation higher than that of the low expectancy group 
even though the true intellectual capacity of the two grouns was equal. Rated 
social competence of the high expectancy group was ETS. = .75, greater than the 
low expectancy group. 

Fourteen studies in the Teachei Behavior category yielded 49 effect sizes 
for an average of .32. There were notable differences in the average effect 
sizes within subcategories. For the teacher behaviors of "sustaining feedback" 
or "reinforcement," the average effect size was .01. This was in sharp contra-st 
to the studies mea.suring the "learning opportunities" provided to the high and 
low expectancy group {ITS. = 1.09). The tendency of teachers to "withdraw from" 
.,or_!!jqnore"_.stude|^^^^ was greater for the low expectancy group (ETS'. = .52).^ .. The 
amount of "attention paid to" and "time spent" witii'the high expectancy group was 
greater than the low expectancy group (E.S. .= .38). 

About half the teacher behavior- studies were' natural istic rather than con- 
trolled experiments. In the naturalistic studies, one cannot separate the effect 
of student ability (or all other characteristics) from the effect of teacher 
expectancy. However, the average effect sizes for the naturalistic studies and 
the controlled experiments did not differ on the average. 

The effect of labeling or expectancy on student IQ following a period of 
instruction was extremely small. Thirteen controlled studies yielded 25 effect 
sizes, the average of which was .14. Forty-one'effec't sizes from 15 studies 
measuring, the impact of teacher expectancy or labeling on student achievement 



The sign of the effect size was reversed for Effects commonly considered 
deleterious, such as "criticize," "ignore," etc. 
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averaged a .26 standard- dev">2t Son effect. The number of words or concepts 
learned in tutorials showed the greatest effect of teacher expectancy or label 
Lll standard deviation average effect size. Reading achievement was also sub- 
stantially affected (ETs. = ,54) as were "in-class grades" (ETS": = ,42), Mathe- 
matics achievement, in contrast, was affected not at all (E,S, = -,07). .Other 
student effects were negligitfle, except for the effect of teacher expectancy of 
social competency of the oiuudents (ETS. = ,89). 

Ability Grouping 

Grouping pupils into classes of homogeneous ability is relevant to the "pull 
out" program organization in which the/ iseparated classes would be more homogen- 
eous" than the'integrated~ cTass7 "The research Tfterature on ab'iTity grouping is ' 
immense and presents the reader with a confusing welter of contradictory, ten- 
dentious, antiquated and dubious studies. The hundreds of studies bn ability 

,' ■ ■ ' t* 

grouping have been frequently reviewed by scholars*, and the history of the 

reviewers' comments is a record of social ideology projected on a body of evi- 
dence too chaotic in its findings to support conclusions of itself. 

In the twilight of the age of Social Darwinism, reviewers read the ability 
grouping literature as supporting homogeneous grouping: or at least not so 
unfavorable as to discourage widespread hope. 

The experimental studies of grouping which have been con- 
sidered fail to show consistent, statistically or education- 
ally significant differences between the achievement of 
pupils in homogeneous grou^^s and pupils of equal ability in, 
heterogeneous groups. This i^^ilure to realize one of the • 

■ ' ' . \ ' . 

For relatively recant reviews, see Findley and Bryan (1971) and 5org (1566) • 




important advantages claimed for ability grouping is not, 
however, evidence that homogeneous grouping cannot result in 
' increased academic achievement. Neither do the experiments 

show that other claims made for grouping cannot be attained 
under proper organization. There was practically unanimou'i 
agreement found among the tea<:hers involved in the studies, 
that the teaching situation was improved by the homogeneous 
grouping. (Rock, 1929, p. 125) 

At the dawning of the Age of Equal Opportunity, the pastiche of inconsistent 

findings on ability grouping was read. as bearing principally on questions of 

ethnic -group segregation: 

In a very real sense, the extent to which the current 
practice of ability grouping is pfermittedh. to exist in public 
schools represent [sic] the extent to which professional 
educators and governmental agencies sanction subrquality T - 
education in a setting that is charged with the responsibil- 
ity of developing each child to his fullest. 
7- - - Xsposi±o, .(19Z3v.p^-177).^. 

Contemporary feelings and a coaple of key court decisions — run strongly 

against ability grouping. In 1971 in Moses vs. Washington Parish School Board 

(Louisiana), the court^held that 

1. ability testing could not be used in recently desegregated schools to 
form ability groups for instruction; 

2. where ability grouping v^esults in disproportionate numbers of Blacks 
^, in low groups, equal protection considerations arise; 

3. the pupils of Franklintoh Elementary School must" be reassigned to 
heterogeneous racially integrated groups. 

When the ideology is stripped off the research reviews on ability grouping, 

one undercurrent is clearly seen: 

So far as achievement is concerned, there is no clear-cut 
evidence that hoitogeneous grouping is either advantageous or 
- disadvantageous. * The studies seem to indicate.^that .homogen- 
eous classification, may be effective if ^ccompe^riied by pro-, 
per adaptation in methods and materials.- 

Miller and Otto (1930, p:-102) 
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It is, ^perhaps, rather futile to take merely the administra- 
tive steps of segregating the slow pMpils unless the super- 
visory program includes constant efforts to improve and 
differentiate the content and techniques of teaching in each 
course of study for the different ability levels. 

Billet (1932, p. 120) 

The results of ability grouping seem to depend less upon 
the fact of grouping itself .than upon the philosophy behind 
the grouping, the accuracy with which grouping is made for 
the purposes intended, the differentiations in content, 
method, and speed and the technique of the teacher,, as well 
as upon more general environmental influences. 
: / : (^ornbll (1935, p. 302) 

No consistent pattern for the effectiveness of homogeneous 
grouping was found to be related, to age, ability level, 
course contents, or method of instruction. 

Inability to control the type of te5(ching and failure to 
provide differentiation of teaching according to ability 
levels are important weaknesses in most of these studies. 
_ _ ' Ekstrom (1959a, p. D 

Ability grouping in itself does not produce improved achieve- 
ment in children. Improved achievement seems rather to 
result from' the manipulation of other complex factors: 
. curriculum adaptation, teaching methods, materials,"^ ability 
of the teacher to relate to children, and other subtle var- 
iables. Hash (1961, p. 430) 

• y • * - 

Ability grouping is inherently neither good nor bad. It 
. is neutral. Its value depends upon the way in which it is 
used. Goldberg, Passow, and Justman (1966, p. 168) 

These are not truisms, altfjpugh they may appear to be so. What thes.e 
reviewers are saying, in effect is that, the research shows homogeneous vs. 
heterogeneous grouping noit to be an important variable in itself . Only as 
ability grouping relates \o or potentiates other changes in instructional acti' 
ities is it a circumstance worth noting. And the accumulated research litera- 
ture of several decaaes shows such an unstable pattern of what takes place 



between teachers and pupils in homogeneous as opposed to heterogi^neous classroorns 
that no generalizations are possible (or useful) about the effects of ability 
grouping.* 

e. 

In spite of the disappointing message that can be extracted from the ability 
grouping 1 iterature, one study impressed us as highly pertinent and credible. 
The study, as yet unpublished, gave evidence of what might occur when ability 
grouping was reversed, as for example, when pupils once "pulled out" arB replaced 
in higher achieving classes. And the study seemed to relate ir< important ways to 
the teacher expectation literature whose message was clear and credible. 

Tuckman and Bierman (1971) conducted ^ controlled experiment on the effects 
of transferring pupils across ability group . Ifnes. Over 350 black junior ajrid 
senior high school pupils were arbitrarily assigned to the next higher ability 
/group than indicated by their test? scores and previous teacher recommendations. 
A randomly equivalent group of pupils of the same size was left in the recom- 
mended lower ability groups. After only a single semester, the results showed 
dramatic effects of the integration of pupi?^s into higher ability classroomsl; 
Standardized test results were generally superior for the artificially acceler- 
ated pupils; but more importantly, teachers recommended that 54% of the elevated 
pupils be retained in the higher ability group whereas on 1% of the comparable 
controls were recommended for the higher ability group! 

* Although we are uncertain whether it ought to be interpreted as a comment on" 
the consistency of lideology or of empiric^J research, it is perhaps worth 
noting that virtually no modern writers cla^im that ability grouping benefits, 
low achievlhg pupils and many claim it is detrimental for them. 
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Peer Tutoring ' - . 

the jDhenomenon of peer tutoring is probably less relevant to the '*pull out" 
problem than any area of colateral research we have considered. However, it is 
true that instruction of compensatory pupil's in the "pull out" setting will 
reduce the opportunities for faster pupils tutoring slower pupils that could 
exist in integrated classes. To the extent that peer tutoring is a part of 
regular classroom instruction, its benefits could be foregone in a "pull oul'^. 
organization. ^ * 

The benefits of peer tutoring have been clearly established. In research 
corrpleted by Dr. Susan S. Hartley at the Laboratory of Educational Research of 
the University of Colorado in June 1977, the facilitative effects of |.eer 
tutoring in mathematics were convincingly documented. Nearly thirty experiments 

some of them involving Title I programs ~ were found that evaluated the 
effects of peer or cross-age tutoring in mathematics. When the 73. outcome mea- 
sures from these studies were integrated statistically, it we^s found that the 
tutored groups scored .60 standard deviations higher in math achie\(ement than the 
nori-tutored groups. What makes this finding even more impressive is that the * 
beneficial effects* of tutoring were greatly superior to the effects of much more 
e.:pensive instructional methods such as computer-assisted instruction, individu- 
alized learning packets and programmed instruction -(see Table 4.2). 

Desegretation ^ . . , 

Almost none of the huge number of studies of the effects of racial desegre- . 
gation or ethnic-group segregation can be regarded as relevant to the "pull out" 
issue. The. principal shortcoming of the desegregation research is that racial or 
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' T^ble 4.2 
Average Effect Sizes (IS - (l^^^^ - l^^^ )/s^ 
By Instructional technique 



•Mean Effect Size 

Standard Deviation 
of Effeci, Sizes 

Standard Error 
of Mean 

Number of 
Effect Sizes 

Number of Studies 



.597 
,684 

.080 

73 
29 



Technique 



CompuW- Individualizeid 
' Assisted ' Learnirig Programmed 
Tutoring Instruction Packets Instruction 



.409 

.588^ 

.062 

89 
35 



.158 
.647 

.055 

139 
51 



.110 
.702 



.074 



89 
41 
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ethnic group composition of schools instead of classrooms is studied. "Pull out" 
programd cIq not alter the racial mixture of schools, though they may and probably 
do change the racial composition of classes within 'schools . 

Only. one study was found in which the impacts on pupils* achievement of 
school and classroom racial composition were separately assessed, McPartland 
(1969) reanalyzed the Coleman data controlling alternately for ''percent white" in 
the classroom and the school. First, he partialed out the effects of pupils* 
family backgrounds and the "percent white" in the school. Then he observed 
cha.rges in blade pupils' Verbal achievement as one moved across the following 
four 'tategon'es of classroom racial composition: 

1) No white classmates 

2) Less than half white^ classmates 
* 3) About half vyhite classmates 

4) More than half white classmates 

On the average, there was a +\16 standard deviation difference from one 
category of classroom racial composition to the next higher category. Put 

dlf ferently_v^^^t two-thi rds standard deviation di f f erence in 

verbal achievement ^etwe'jn blacks in^all black classes and blacks^in classrooms 
more than half white even after pupils' family background and racial composition 
of school are statistically controlled.* The fact remains that McPartland 's 



In the NIE (September 1977) report, it was concluded that studies of peer 
influences do not support the claims made by proponents of mainstreaming that 
the socioeconomic and achievement levels of their classmates affect pupils' 
academic performance" (p. 7). However, the authors apparently overlooked 
McPartland's study and cited Smitii4ia7^-4;i--.^^^^ In 
fact. Smith (1972) pointed out that technical problems in the analysis of the 
Coleman data resulted in the claim of peer group influence not being properly 
tested; he did not write that the claim had been refuted^ 
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study was non-experimental and it leaned heavily on imperfect statistical adjust- 
ments (e.g., controlling family background would not control perfectly for abil- 
ity tracking taking place within the school). In spite of the inevitable imper- 
fections of non-experimental research of which McPartland's study partakes its 
share, the study is nonetheless suggestive (if only of a foregone conclusion, a 
skeptic might argue). , 

Other research (equally lacking in experimental rigor and resting heavily on 
the sociologist's faith in ex post facto statistical control ). suggests that a 
pupil's classmates are more influential thc^n his mere schoolmates. Campbell and 
Alexander (1965) and McDil 1 , Meyers , and Rigsby (1966) purport to demonstrate 
that a pupil's close friends have more effect than his schoolmates on his values, 
attitudes and educationaT aspirations . 

These few research studies suggest that as "pull out" alters the immediate 
classroom social composition (the race, ability and achievement of a pupil's 
classmates), it, is altering a significant set of influences of his own academic 
achievement, values, and aspirations. And it appears that the imnediate environ- 
ment of the "pulled out" pupil is less favorable in these respects than that of 
the compensatory education pupil left in the regular classroom." 

Sutmiary of Effects of "Pull Out" Inferred from Related Research 

I. ' 

Whether the related research reviewed here offers a useful analogy for "pull 
out" programs depends on how closely such programs resemble mainstreaming the 
-handicapped, ability grouping, racial desegretation, etc. To be sure, the "pull 
out" technique shares gross features with each ^of the major topics reviewed in 



this section. The low achieving "Title I eligible" pupils are "pulled out," and 
in this respect "pull out" is like ability grouping. Ethnic group differences in 
achievement being what they currently are, "pull out" programs result in a.mild 
form of ethnic group segregation --although no one would claim that the intent 
behind them is discriminato r. Separating high and low achieving pupiVs lessens 
opportunities for peer tutoring and role modeling, but it may provide opportun- 
ities for specialized instruction. "Pulling out" disadvantaged^pupils, may. rein- 
force a form of labeling and create expectations for failure in the minds pf 
•lrea"chers and other pupils; or it may.xreate the expectation that the pupils 
"pulled out" will prosper since special efforts are being made in their behalf. 

The difficulty in drawing an analogy between "pull out" and allied forms of 
instruction is that too little is known about exactly what happens to pupils who 
are "pulled out" and what they, their peers, and their teachers think about it. 
Only the' crudest demographics of the phenomenon are known (cf. Chapter 3), and 
they are insufficient t^^. indicate which areas of related research are most perti- 
nent and how confidently one could argue that their findings (e.g., of main- 
streaming, or of ability grouping) apply with equal force to "pull out." 

Nevertheless, research distantly related to "pull out" cannot bfi ignored. ^ 
We find that, in general, research does not support the wisdom of instructi.on 
under conditions like those that prevail in "pull out" programs. Pupils pulled 
out of regular classrooms would have to receive remarkably effect ivejcom^^ 
programs to offset the potential risks incurred. In our opinion, the "pulled, 
out" pupil is placed in moderate jeopardy of being dysf unctionally labeled, of 
roregoing opportunities for peer tutoring and role modeling, and of being segre- 
gated from pupil s> of different ethnic groups. 
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Chapter 5 

HISTORICAL AND POLITICAL CONTEXT OF "PULL OUT" 
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... We are not historians, much less political scientists. Our knowledge of the 
politics of education is like the knowledge that a blocking duniny has of football 
strategy or a punching bag of the art of self-defense. Nevertheless, our reading 
Bf several documents on the creation and administration of Title I, interviews 
with a few dozen persons at all levels concerned about compensatory education, 
and the senior author's experience in Title I evci.Tliation prior to 1970 have left 
us with deep impressions about how the "pull out" problem was created, what sus- 
tains It, and what should be done about it. And the opinions that fol low have • 
been tested for consistency where pbs sib] ^ the data in the three previous 

chapters. ; ' 

Congr^iional debate on ESEA 1965 broached at least two significant issues: ' 
1) the question of general versus categorical aid; and 2) the church-state issue. 
The fear was strong in many quarters that Federal aid to schools would be used as 
general aid, spent mP'-^ly reduce local taxes and not for the special purposes 
embodied in the tit of ES£A. Eventually,, the "categorical a id^^s proponents : 
won. The church-state issue was finessed by specifying that Tit'l% I aid was to 
pupils not to schools — the latter distinguishable as either publ it or parochial, 
~"~fhe~f6rmer"apparently not. 

By the time the. bill became P.L. 89-10, USOE was strongly predisposed (as a 
. re'sult of the Congressional battles and a natural bent) to write regulations , that 
insured that Title I funds would be spent only on "eligible" pupils for compensa- 
tory programs. Thus emerged regul ations .on "supplementing, not supplanting," 
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"excess cost," etc. The five-year period immediately following enactment of ESEA^ 

\ ■ 

1965 saw the fires of enthusiasm dampened by a succession of unfavorable evalua- 
tlon reports. Questions of the validity of these evaluations aside (and serious 
questions remain) , the reports were disturbing to USOE Title I personnel. They 

tended, in general , to believe the evaluations accepting top credulously, in 

. ■ ■ * . \ ■ 

our opinion, the validity of the outcome measures used and ascribe the nega- 

'\ ■ " 

tive findings to the failure of the public schools to "target" Title I funds in 



\ 

sufficient quantities on low achieving pupils. It Was widely decried that the 



average compensatory education pupil received only about $100 of Title Iservices 



in a year. 



has been no 



From the inception of the idea of Title I to the present, there 
strong defender of the position that all pupils in poor schools dese»fve compensa- 
tory services • The argument can be advanced with some force that any pupil in a 
school with a high concentration of poor children is put at an educational disad- 
vantage in numerous ways: (1) his school cannot attract as experienced or compe- 
tent teachers as a rich school; (2) he lacks the advantages of social relation- 
ships with peers who would„ provide good role models; (3) he may attend classes 
that are disrupted or disorganized by "the special plrobTfems that poor or low 
achieving pupils present; etc. Considering these secondary or indirect effects 
of concentrations of poor pupils io schools, it is clear that even pupils of 
above average achievement have a right to compensation of some educational disad- 
vantages. But the debate in Congress that might have reached such issues became 
trivialized. The spectre of the rich kid in Montgomery County, VirginiajWho 
might receive Title I services was repeatedly invoked by the defenders of 



categorical aid to put down the proponents of general- aid. In the pitched battle 
over categorical versus general aid, sight was lost of the middle ground. Title 
I funds could have been aimed at schools with large concentrations of poor chil- 
dren, but with the understanding that (1) all pupils in poor schools need help 
because of the direct and indirect effects'of poverty, and (2) school programs 
- ant^ schooV-orgarriza1:ioR-^re totTririrs "tha^^ 
intervention Without risk of detrimental effects to the school and its pupils. 

However, this second guessing of history does not change the fact that by 
1970, USOE was prepared to tal^e vigorous steps to see that Title I funds were, 
"targeted" on eligible pupils (i.e., pupils with low achievement test scores). 
The mechanisms of enforcing targeting were audits of programs for supplanting 
violations and the training of Title 1 officials in state education agencies, to 
encourage targeting. When one listens to SEA Title I coordinators in different 
states, one hears a single message that speaks of a common origin. The SEA Title 
I coordinators tell the local school districts that they must institute a 
"recognizable" compensatory education program in drder to avoid a "supplanting 
violation." They advance a common set of arguments in an -insensitive casuistry 
that local schools are at a loss to rebut. When the local schools wish to remove 
a low achieving pupil from regular reaxling (where he. isn't benefiting) to reme- 
dial reading, the SEA Title I coordinators claim that such a move would consti- 
tute a isupplanting violation.* If the LEAs insist that their Ti' eliaible 

* ■'■ 

Thus one sees that the supplanting issue, that can only be interpreted as an 
^ economic concern about replacing local taxesswith Federal taxes in the 
Congressional debates on ESEA 1965, has been translated into an educational 
concern by 1975, and the SEAs are using it to second guess the curricular 
decisions of the LEAs. 



pupils cannot profit frorn the reading instruction in regular classes, the SEA 
coordinator chastizes them for running programs normally that do not benefit all 
pupils. When the LEAs ask how they can avoid a supplanting'violation , the SEA 
coordinator offers the "pull out" model: all pupils should receive basal reading 
for thirty minutes, then the Title I eligible pupils should be removed to a 
special room for remedial (Title I) instruction by a reading specialist while the 
non-eligible pupils in the regular classroom can be given study time, science, or 
social studies. By some chop logic, supplanting is not supplanting at all if 
what is supplanted is science or social studies. The SEA Title I coordinators 
are purveyors of sophistry, and bad /sophistry at that. 

We give credit (or lay blame) for the widespread use of the "pull put" model 
to the USOE Title I regulations and their enforcement by theSEA Title I coordin- 
ators. . As one state education department official put it" "'Pull out* exists 
for one reason only; because the 'locals' are afraid Big Brother will catch them 
in a 'supplanting' violation." The "pull out" model is an artifice designed to 
demonstrate to outsiders compl iance with Title I regulations. It was/not devised 
through the exercise of professional judgment on the questions of how best to 
compensate for the disadvantages suffered by pupils in poor schools. 

The "pull out" model is advocated only by a bureaucratic constituency in 
USOE and the SEAs. It is not supported on its educational merits by any other 
significant groups: not by researchers, not by local administrators, not by 
teachers individually or en masse . The National Education Association regards 
"pull out" as a minor issue. NEA feels that in general supplementjary services 

1j ■ - ■ 



- 45 - 



should be rendered as far as possible in the regular classroom. Their primary 
concern is wi':h pupil -to-teacher ralios; they would watch to see that "main- 
streaming" Title I pupils did not increase pupil-to-teacher jatios, but thsy 
v/buld not be greatly concerned otherwise. The "pull out" problem seems to be 
no one's major concern. But it may well be one of those quiet .inconspicuous 
matters that count heavily in ways seldom clearly seen. 
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Chapter 6 

CONCLUSIONS, OBSERVATIONS, AND RECOMMENDATIONS 

Our work has led us to the following conclusions and observations about the 
"pull out" technique and several reconmendations for dealing with the problems it 



raises. 

1. Pulling Title I eligible pupils out of regular classrobms for compensa- 
tory instruction is virtually uni'Ni^ersal . 

2. The "pull out" procedure £er se^ has noj.qlear academic or social benefits 
and may, in fact, be detrimental to pup^ils' progress and adjustment to 
school 

37 The "pull out'*^ procedure is used by schools more-to~satisfy Title I 

regulations than because it is judged by teachers to be a sensible and 
beneficial plan. 

We wish to bring the following recommendations to the attention of those 
persons at all levels dminisu Ti ; programs and who will influence the 
evolution of compensatpry education: 

1. The Title I regulations, which now reflect an overweening concern with 
targeting funds on "eligible" pupils, should be examined. New consider- 
ations should be given to the needs of all pupils in poor schools ai\d 
the integrity of total >. school programs. 

2. Instructional stratiegies should be devised that would eliminate the 
invidious labeling of compensatory education pupils and their segrega- 

^ tion from classesi^ of /".regular" pupils. ' • . 



3. Teachers, administrators and other persons connected with Title I pro- 
grams should, ^be informed of the findings of research on the "pull out" 
method and associated phenomena. 

4. Methods should be devised of counteracting^ the possibly detrimental 
effects of "pull out" where educators choose to use it or have no 
reasonable alternatives. Such methods could include means for coor- 
dinating instruction across two sites and. techniques of teacher obser- 
vation that lessen the possibility that "pulled out" pupils will be 
unconsciously neglected in regular classes. 
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